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MAJ  Eric  Tollefson,  LTC  Jeffrey  Schamburg  (TRAC-MTRY) 

Mr.  John  Geddes  (REF) 

MORSS  WG28  Presentation 
14  June  2007 


Agenda 


Purpose  of  this  briefing 

To  describe  TRAC-MTRY  analytical  support  to  the 
Rapid  Equipping  Force  (REF). 


•  Overview. 

•  Overall  methodology  development. 

•  Comparison  of  alternatives  decision  support  tool  development. 

•  Analysis  guidebook  development. 

•  Methodology  applications. 

•  Other  supporting  efforts. 

-  Human  systems  integration  (HSI)  analysis. 

-  Information  technology  management  analysis. 

-  Rapid  ordering  system  (ROS)  integration. 

•  Conclusions. 

TRAC-MTRY:  The  US  Army  Training  and  Doctrine  Command  (TRADOC)  Analysis  Center  (TRAC)  in  Monterey,  CA 
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Research  Overview 


*  Problem  statement:  the  REF  needs  an  iterative,  quick 
turn-around,  standardized  systems  analysis 
methodology  that  considers: 

-  Needs  and  capability  gaps. 

-  Comparison  and  evaluation  of  alternative  solutions. 

-Tactics,  techniques,  and  procedures  (TTPs). 

-  Lifecycle  costs  and  logistical  considerations. 

-  Distribution  of  capabilities. 

*  Research  objectives. 

-  Develop  an  iterative,  quick  turn-around,  standardized 
systems  analysis  methodology  with  supporting  tools  and 
analyses. 

-  Demonstrate  their  application  on  a  REF  system  under 
consideration. 

-  Refine  and  deliver  the  methodology  and  supporting  tools. 


14  June  2007 


REF  Analytical  Support  MORSS  Briefing 


Team  Overview 


*  Project  being  conducted  as  a  collaborative  effort  within  a 
multi-disciplinary  team. 

•  Team  participants. 

-  MAJ  Eric  Tollefson,  lead  analyst  and  project  lead. 

-  Mr.  Steve  Pranger,  supporting  analyst. 

-  LTC  Jeff  Schamburg,  senior  analyst. 

-  LTC  John  Willis,  former  TRAC-MTRY  analyst. 

-  Mr.  Eric  Johnson,  TRAC  Methods  and  Research  Office  (MRO). 

-  Dr.  Gene  Paulo,  Systems  Engineering  (SE)  team  lead. 

-  Dr.  Tom  Housel,  Information  Science  team  lead. 

-  Dr.  Ron  Tudor,  Business  and  Acquisition  team  lead. 

-  COL  (Ret)  Larry  Shattuck,  Human  Systems  Integration  (HSI) 
and  Operations  Research  (OR)  team  lead. 

-  NPS  students. 

-  7  thesis  students  (business,  OR/HSI,  information  sciences). 

-  9  class  project  students  (SE,  information  sciences). 

-  Mr.  John  Geddes,  GS-15,  REF  Technology  Management  Director. 

-  MAJ  Robert  Lenz,  REF  analyst. 
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General  Analysis  Methodology 

Key  Characteristics 


Familiar 

Drawn  from  other  Army,  DoD,  and  business  problem 
solving  methodologies  and  best-practices. 

Rapid 

Able  to  be  executed  quickly,  but  thoroughly. 

Intuitive 

Understandable  process  that  requires  minimal  training, 
leverages  current  REF  expertise  and  processes,  and 
uses  easy-to-learn  supporting  tools. 

Modular 

Series  of  analyses  and  tools  that  can  be  added  or 
removed  based  upon  schedule  or  requirements. 

Tailorable 

Flexible  analyses  and  tools  that  can  be  modified  by  the 
analysts  to  match  their  skills  and  experience,  or  refined 
based  upon  lessons  learned. 

Iterative 

Analyses  that  can  be  repeated  and  updated  as  new 
information  becomes  available. 

Scalable 

Can  be  executed  at  differing  levels  of  detail. 
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General  Analysis  Methodology 

Primary  Processes 


PROBLEM  DEFINITION 

1.0  In-Theater  Need  Identification. 

2.0  Requirements  Analysis  /  Project  Validation. 

3.0  Technical  Course  of  Action  (COA)  Analysis. 

4.0  Initial  Project  Decision  Briefing. 

SOLUTION  DEVELOPMENT  AND  ANALYSIS 
5.0  Solution  Set  Refinement  and  Analysis. 

6.0  Initial  Implementation  Plan  Development. 

7.0  Implementation  Plan  Approval  Decision  Briefing. 

IMPLEMENTATION  AND  TRANSFER 

8.0  Implementation  Plan  Refinement  and  Execution. 

9.0  Fielded  System  Evaluation. 

10.0  Fielded  System  Transfer. 

Refined  based  on  applications  feedback,  REF  process  assessment  work, 

and  mapping  to  REF  process. 


14  June  2007 


REF  Analytical  Support  MORSS  Briefing 


7 


General  Analysis  Methodology 

Flow  Diagram 
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Analysis  Guidebook 

Overview 


Purpose:  to  provide  REF  analysts  a  quick  reference  guide  that  describes  the 

overall  REF  analysis  process. 

•  Concept. 

-  Develop  an  easy-to-understand  and  succinct  reference  guide  for  REF 
analysts. 

-  Identify  key  aspects  of  each  analytical  process  and  sub-process. 

-  Suggest  potential  tools  and  techniques  to  support  REF  analyses. 

•  Description. 

-  Maps  the  inputs  and  outputs  of  each  process  and  decomposes  each  of 
the  processes  into  sub-processes. 

-  Briefly  describes  the  inputs  and  outputs  of  the  processes. 

-  Briefly  describes  of  each  of  the  sub-processes. 

-  Identifies  potential  supporting  tools  and  techniques. 

•  Developed  in  MS  PowerPoint  in  an  easily-referenced  format. 

•  The  next  two  slides  show  an  example  of  the  input-to-output 
mapping  for  the  first  analytic  process  and  an  example  of  its 
decomposition  into  sub-processes. 


The  analysis  guidebook  will  provide  all  REF  analysts,  regardless  of  their 
analytical  background,  a  common  framework  for  REF  analyses. 
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1.0  In-Theater  Need  Identification 

Inputs  to  Outputs 

INPUTS  PROCESS  OUTPUTS 
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1.0  In-Theater  Need  Identification 

Sub-processes 


1.2  Identify  System  Characteristics  and 
Potential  Solutions 


1.4  Identify  Support  and  Procurement 
Concepts 


(and)-^ 


1.4.1 

Identify 

Procurement 

Objective 


1.4.3 

Conduct  Initial 
Coordination 


1.4.2 

Identify  Support  I—1 
Requirements 


1.4.4 

Recommend  a 
Purchase  Plan 
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2.0  Requirements  Analysis  /  Project  Validation 

Inputs  to  Outputs 


INPUTS 

PROCESS 

OUTPUTS 

A.1  Need 
Document 


(10-Liner,  ONS, 
JUONS) 


C.2 

TM  Potential 
Solutions 


H 


Technical  COA 
Analysis  Results 


2.0 

Requirements 
Analysis  /  Project 
Validation 


J 

TM  White  Paper 


K 


TM  Board  Slide  / 
Briefing 


A.2 

QA’d  Need 
Documerrt 

B.1 

Requirements 
Quad  Chart 

C.1 

Partial  Solution 
Set___ 

D 

REF  Priority 

'  "  "  . 

E 

Supporting 

Requirements 

Information 

F 

Requests  for 
Information 
(RFIs) _ 

G 

Correspondence 

History 

Database  Entries 
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2.0  Requirements  Analysis  /  Project  Validation 

Sub-processes 


2.1  QA  Need  Document 
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2.0  Requirements  Analysis  /  Project  Validation 

Discussion  (Slide  1  of  4) 


•  Purpose:  To  transform  an  initial  need  identification  into  a  thorough  set  of  requirements  by  refining 
the  initial  need  documentation  and  conducting  a  detailed  mission  (need)  analysis. 

•  Rationale:  As  with  the  initial  identification  of  the  need,  this  essential  step  in  defining  the  problem 
and  determining  the  requirements  is  critical  to  the  success  of  the  project.  The  requirements 
identified  in  this  process  is  the  foundation  upon  which  the  ultimate  recommendation  of  a  solution 
and  the  implementation  plan  will  be  based. 


•  Inputs:  The  primary  input  to  this  process  is  a  10-Liner,  Operational  Need  Statement  (ONS),  or  a 
Joint  Use  Operational  Need  Statement  (JUONS).  Inputs  could  also  include  documentation  from  the 
REF  Technology  Management  team  as  they  vet  ‘good  ideas’,  new  technologies,  insert 
technologies,  or  command-directed  technologies. 


•  Sub-process  Discussion: 


-2.1  QA  Need  Document:  In  this  step,  the  team  receives  a  need  identification  from  the  field  (10- 
Liner,  ONS,  JUONS)  and  reviews  it  for  completeness.  If  the  documentation  is  missing  critical 
information,  the  team  returns  it  to  the  originator  for  clarification.  Some  information  is  not 
required  to  be  complete  in  the  Need  Document  (e.g^  organizational  concept,  procurement 
objective,  support  requirements,  and  availability). 'The  output  is  a  QA’d  need  document. 


-2.2  Conduct  Mission  Analysis:  This  is  a  very  complex  step  and  includes  numerous  sub-steps  to 
ensure  that  the  initial  need  statement  is  accurately  transformed  into  a  thorough  set  of 
requirements.  A  discussion  of  these  sub-steps  is  essential  to  understand  this  process. 


-  2.2.1  Stakeholder  and  Background  Analysis:  The  team  begins  this  step  immediately  and 
continues  to  execute  it  throughout  the  entire  process.  A  stakeholder  can  be  defined  as 
anyone  with  a  vested  interest  in,  or  influence  upon,  the  problem.  The  team  will  identify  and 
interview  stakeholders  for  almost  every  aspect  of  Mission  Analysis.  Because  time  is  a  limited 
commodity  for  all  stakeholders,  the  team  must  put  thought  into  the  information  they  need 
before  they  begin  their  interviews.  Key  stakeholders  win  include  the  unit,  threat  subject 
matter  experts  (SMEs),  other  operational  SMEs,  REF  personnel,  project  managers  (FTVIs), 
maintenance  and  sustainment  SMEs,  industry,  and  other  personnel  who  can  help  provide  or 
clarify  required  information.  Additionally,  the  team  will  check  with  DAMO-CIC  to  determine  if 
they  are  working  on  a  similar  ONS  prior  to  an  AR2B.  In  conjunction  with  interviewing 
stakeholders,  the  team  will  also  conduct  background  research  to  clarify  their  information. 

The  key  outputs  of  this  step  are  a  correspondence  history  and  the  information  provided  by 
the  stakeholders  and  through  the  background  research.  In  some  cases,  if  a  PM  solution  has 
been  identified  to  meet  the  need,  this  process  can  end  with  transition  to  the  PM. 
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2.0  Requirements  Analysis  /  Project  Validation 

Discussion  (Slide  2  of  4) 


•  Sub-process  Discussion  (continued): 

-2.2  Conduct  Mission  Analysis  (continued): 

-  2.2.2  Capability  Gap  Analysis:  The  team  will  refine  the  capability  gaps  identified  by  the  REF 
FWD  team  (if  provided)  and  identify  applicable  sub-capability  gaps  as  well.  The  team  should 
try  to  identify  the  frequency  with  wnich  the  gap  occurs  and  tne  impact  of  the  gap  on  the  unit. 

-  2.2.3  Operational  Environment  Analysis:  The  team  will  describe  the  operational  environment 
in  which  the  solution  will  be  used,  identify  ROE  and  other  legal  considerations,  and 
understand  the  threat  and  how  the  threat  might  defeat  and/or  adapt  to  the  employment  of  the 
solution. 

-  2.2.4  Operational  Opportunity  and  Context  Analysis:  This  analysis  captures  how  the  potential 
solution  may  be  employed,  in  terms  of  tactics,  techniques  ana  procedures  (TTPs),  identifies 
potential  drawbacks/penalties  of  the  system,  identifies  other  systems  with  which  the  potential 
solution  will  have  to  interoperate,  identifies  how  often  the  potential  system  will  be  available  to 
the  unit  for  use,  and  how  often  situations  will  occur  for  which  the  system  will  provide  benefit. 
The  latter  two  analyses  help  the  REF  determine  if  the  fielded  solution  will  actually  provide 
significant  benefit  to  the  unit,  versus  only  occasional  benefit.  These  results  should  be 
balanced  with  the  frequency  and  impacts  of  the  gap  itself  to  help  the  REF  prioritize  the 
current  requirement  with  other  competing  requirements. 

-  2.2.5  Fielding  and  Sustainment  Analysis:  This  analysis  captures  the  number  of  systems  that 
are  required  and  whom  they  will  be  fielded  to  (basis  of  issue  -  number  by  echelon),  the 
expect  logistical  footprint  required  to  support  the  system  (manpower,  spares,  facilities, 
special  equipment)  and  an  initial  estimate  of  the  lifecycle  cost  (procurement,  maintenance, 
sustainment,  etc.)  of  a  potential  solution  (to  help  identify  the  funding  source)  and  an  estimate 
of  the  fielding  schedule. 

-  2.2.6  Operational  Feasibility  Analysis:  This  analysis  captures  the  organization,  training,  leader 
education,  and  personnel  impacts  of  fielding  the  system  and  includes  the  human  factors 
issues  (comfort,  usability,  etc.)  associated  with  the  fielded  systems’  impacts  on  the  users, 
leaders,  and  maintainers. 

-  2.2.7  Functional  Analysis:  This  analysis  defines  in  greater  detail  the  required  functions  of  the 
system  in  terms  of  what  the  system  must  do  in  order  to  mitigate  the  capability  gap  and  satisfy 
the  unit’s  need.  This  is  primary  step  for  defining  the  requirements  of  tne  potential  materiel 
solution  and  provides  the  basis  for  alternative  identification  and  the  evaluation  framework 
that  will  be  used  to  compare  the  alternatives.  It  is  important,  when  defining  the  functions,  to 
define  them  independently  of  potential  solutions  already  being  considered? 
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2.0  Requirements  Analysis  /  Project  Validation 

Discussion  (Slide  3  of  4) 


•  Sub-process  Discussion  (continued): 

-2.2  Conduct  Mission  Analysis  (continued): 

-  2.2.8  Evaluation  Framework  Design:  This  step  should  flow  directly  from  the  functional 
analysis  and  the  stakeholder  analysis.  The  objectives  of  the  stakeholders  should  be 
combined  with  the  required  functions  to  define  the  desired  performance  by  function.  The 
team  will  then  derive  criteria  from  those  objectives  that  will  be  used  to  measure  and  evaluate 
alternative  performance.  Criteria  come  from  the  system  attributes  defined  in  the  10-Liner, 
measures  suggested  by  stakeholders,  and  other  measures  developed  by  the  team.  Criteria 
must  be  developed  in  such  a  way  that  is  not  specific  to  a  particular  solution  so  that  they  can 
be  used  to  evaluate  any  potential  alternative.  Criteria  can  either  be  natural  (a  clearly  defined 
and  understood  scale}1ike  pounds)  or  constructed  (created  specifically  to  measure  the 
performance  of  a  particular  function/objective,  such  as  a  5-star  rating  for  safety).  Once  a  list 
of  criteria  has  been  established,  the  team  must  weight  the  criteria  according  to  the 
importance  of  the  primary  stakeholders,  so  that  comparisons  can  be  made.  The  evaluation 
framework  will  be  refined  and  completed  later  by  the  Project  Officer. 

-  2.2.9  Initial  Alternative  Identification:  In  this  step,  the  team  will  compile  a  set  of  potential 
solutions,  which  will  come  from  the  unit,  other  stakeholders,  and  other  sources.  The  team 
should  consider  non-materiel  alternatives  within  the  DOT_LPF*  domains  as  well. 

-2.3  Assign  REF  Priority:  The  requirements  team  should  assign  a  priority  (routine,  essential,  or 
urgent)  fo  the  requirement  based  primarily  upon  unit/REF  FWD  recommendation,  but  also  upon 
the  Mission  Analysis  conducted  as  part  of  this  process. 


*DOT_LPF:  Doctrine,  Organization,  Training,  Leader  Education,  Personnel,  Facilities  (Materiel  omitted) 


•  Potential  Tools  and  Techniques: 

-Stakeholder  Analysis:  The  key  to  the  success  of  this  process  is  the  interaction  between  the  REF 
analysis  teams  and  key  stakeholders. 

-TRADOC  ARCIC  Current  Force  Capability  Gap  Documentation:  Provides  a  description  of  the 
currently-approved  capability  gaps,  sub-capability  gaps,  and  potential  solutions  for  mitigating 
the  gaps. 

-Human  Systems  Integration  (HSI)  Tool:  Tool  developed  by  TRAC-MTRY  to  assist  in  the 
identification  of  potential  HSl  ana  DOTMLPF  issues  associated  with  fielding  a  materiel  solution. 

-System  Decomposition:  A  technique  to  decompose  the  required  solution  into  its  component 
parts,  structure,  and  functions.  This  is  a  good  technique  for  gaining  an  understanding  of  the 
system  and  its  key  elements. 
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2.0  Requirements  Analysis  /  Project  Validation 

Discussion  (Slide  4  of  4) 


•  Potential  Tools  and  Techniques  (continued): 

-System  Context  Diagram:  A  technique  used  to  place  the  system  and  its  sub-systems  in  the 
context  of  the  operational  environment,  which  is  helpful  for  identifying  interoperability  issues, 
potential  TTPs,  and  echelon  of  system  employment. 

-Legal  Advice:  Army  lawyers  can  help  identify  unique  legal  requirements  for  certain  types  of 
systems  under  consideration. 

-Functional  Decomposition:  Techniques,  such  as  functional  hierarchies  and  functional  flow 
diagrams,  used  to  decompose  the  need  into  required  functions. 

-Objectives:  A  method  of  thinking  that  translates  required  functions  into  desires  concerning 
performance  that  can  help  lead  directly  into  the  criteria  or  measures  for  alternative  evaluation. 

-Value  Measures:  Techniques  for  developing  criteria  to  capture  the  value  of  the  key  aspects  of 
alternative  performance  for  evaluation  and  comparison  purposes. 

-Value  Hierarchy:  Technique  used  to  capture  the  evaluation  framework  that  starts  with  the 
functions  and  flows  into  objectives  and  then  criteria,  or  value  measures. 


•  Outputs: 

-QA’d  Need  Document. 

-Requirements  Quad  Chart:  Pre-defined  REF  document  that  includes  a  succinct  description  of 
the  problem  (this  should  be  a  more  complete,  revised  problem  statement  that  builds  upon  the 
initial  problem  statement  included  in  the  Need  Document),  a  photo  of  the  potential  solution  (if 
identified),  a  description  of  the  system  (either  explicitly,  if  a  solution  has  been  identified,  or  in 
terms  of  requirements  if  a  solution  has  not  been  identified),  the  applicable  TRADOC  ARCIC 
Current  Force  capability  gap(s)  and  sub-capability  gap(s),  proposed  and  recommended  courses 
of  action  for  the  procurement  of  the  solution,  the  requesting  activity,  identification  of  similar 
items  REF  may  have  considered  in  the  past,  an  initial  sustainment  strategy,  and  a  potential 
vendor  (if  identified). 

-Partial  Solution  Set:  A  set  of  potential  materiel  solutions  described  in  an  Excel  spreadsheet. 

-REF  Priority. 

-All  of  the  supporting  Requirements  Analysis  /  Project  Validation  products. 

-Unanswered  Request  for  Information  (RFIs). 

-Stakeholder  Correspondence  History:  A  log  of  the  critical  discussions  between  the 
requirements  team  and  key  stakeholders. 

-Required  Database  Entries. 


14  June  2007 


REF  Analytical  Support  MORSS  Briefing 


17 


Comparison  of  Alternatives  Decision  Support  Tool 

Overview 


Purpose:  to  develop  a  comparison  of  alternatives  decision  support  tool  for  rapidly 
and  quantitatively  researching,  analyzing,  and  selecting  the  best  materiel  solution. 

•  Concept. 

-  Provide  the  user  a  rapid  means  to: 

-  Enter  criteria  and  alternative  data. 

-  Weight  the  criteria. 

-  Score  the  alternatives. 

-  Present  the  user  with  analysis  products  to  facilitate: 

-  Alternative  comparison. 

-  Cost-benefit  analyses. 

-  Sensitivity  analyses. 

•  Prototype  Decision  Support  Tool. 

-  Uses  Microsoft  Excel  as  the  primary  platform. 

-  Provides  user-friendly  interface  and  performs  actions  via  Excel 
formulae,  ActiveX  controls,  and  VBA  code. 

-  Plan  to  provide  both  simple  and  complex  weighting  and  scoring 
methods. 


A  decision  support  tool  will  facilitate  rapid,  quantitative  alternative 
comparisons  while  considering  critical  evaluation  measures. 
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General  Analysis  Methodology 

Applications 


•  Initial  case  study,  conducted  in  Fall,  2006. 

-  Notional  system  under  consideration:  improved  body  armor. 

-  Lessons  learned. 

-  Quantitative  analysis  can  be  valuable  in  making  a  quality  decision. 

-  Tools  and  techniques  provided  to  the  REF  analysts  must  be  easy-to-learn 
and  intuitive,  not  requiring  extensive  analytical  oackground. 

-  Varied  nature  of  the  problems  and  materiel  solutions  that  REF  analysts  will 
consider  require  flexibility  in  the  design  approach. 

-  Stakeholder  input  is  essential,  yet  often  difficult  to  obtain;  useful  to  start 
with  analysts’  opinions  and  then  seek  stakeholder  revision. 

•  Methodology  development  support. 

-  Demonstrated  methodology  tailorability  for  a  particular  project  under 
consideration  at  the  REF. 

-  Delivered  tailored  methodology,  but  did  not  execute. 

•  Limited  case  study  considering  green  laser  alternatives  conducted  in 
Spring,  2007. 

-  Demonstrated  application  of  the  Decision  Support  Tool  (DST). 

-  Led  to  DST  refinements. 

•  Subsequent  case  studies  to  be  conducted  in  June. 

-  Conduct  study  on  a  REF  system  under  consideration. 

-  Execute  the  methodology  from  start  to  finish. 

-  Make  final  refinements  based  upon  lessons  learned  before  delivery  to 
the  REF. 
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General  Analysis  Methodology 

Overview 
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Weapon-mountable  Green  Laser  Application 

DST  Input  Screens 


Criteria 

Criterion  Status 

Criterion 

Importance 

Weight 

Include 

Hiqh 

Output  Powei 

Include 

1  LOW  | 

Be.ini  Divergence 

include 

1  Average 

Rail  Monntahility 

Include 

1  Vary  High  f 

Range 

include 

l  High 

Battery 

include 

1  Low 

Venrloi  Reliability 

include 

Hiqh 

Criterion  8 

Don’t  Include 

'  -•  Select  - 

Criterion  9 

Don’t  Include 

1  -  Select  -  1 

Criterion  10 

Don’t  Include 

1  -  Select  -  | 

Simple  Criterion  Weighting:  The  user 
chooses  criterion  importance:  very 
low,  low,  average,  high,  or  very  high. 


Criteria  Selection  and  Simple  Weighting 


Simple  Alternative  Scoring:  The  user 
evaluates  the  performance  of  each  alternative 
for  each  included  criterion:  not  at  all,  poor, 
adequate,  good,  or  excellent. 


User  chooses  to  include  or  not  include 
criteria  and  alternatives,  giving  the  user  the 
flexibility  to  add  or  remove  criteria  or 
alternatives  throughout  the  process. 


- ^ - - 

Alternatives  *  *"  •«. 

Helios 

Dazzlei 

PD/G-105 

Dazzlei 

Mini  Gieen 

GBD  3 

BTG  2S 

i 

'  Comfort 
<  Class  3 

GL  V5 

S532A 

Alternative 

9 

Alternative  Status  ►  ^  ^ 

include 

Include 

include 

Include 

include 

Include 

include 

Include 

Don’t  Include 

- ^ 

Alternative  Evaluations 

Weight 

Include''  4 

1  "  Poof  —  ’ 

Excelfenf" 

“Good” 

“Mlqaafi  " 

"Adequate" 

0oo  W 

—  Food  -  " 

■  ~ 

Output  Power 

Include 

Excellent 

Adequate 

Adequate 

Good 

Poor 

Poor 

Poor 

Poor 

-  Select  -- 

Beam  Divergence 

Include 

Good 

Adequate 

Excellent 

Excellent 

Adequate 

Adequate 

Adequate 

Adequate 

-  Select  - 

Rail  Mountahility 

Include 

Excellent 

Excellent 

Excellent 

Adequate 

Poor 

Poor 

Poor 

Poor 

-  Select  - 

Range 

Include 

Poor 

Poor 

Adequate 

Excellent 

Good 

Poor 

Good 

Good 

-  Select  — 

Battery 

Include 

Good 

Excellent 

Excellent 

Excellent 

Excellent 

Excellent 

Excellent 

Excellent 

--  Select.  -- 

Venrloi  Reliability 

Include 

Excellent 

Adequate 

Adequate 

Adequate 

Adequate 

Adequate 

Adequate 

Adequate 

-  Select  - 

Criterion  8 

Don’t  Include 

--  Select  - 

-  Select  — 

~  Select  - 

-  Select  - 

-  Select  - 

C  nlorf 
—  u'ClcU  — 

-  Select  - 

H  Select  + 

. Select  - 

Criterion  9 

Don’t  Include 

--  Select  - 

--  Select  - 

--  Select  -- 

-  Select  - 

--  Select  -- 

—  Select  — 

-  Select  - 

-  Select  -- 

--  Select  - 

Criterion  10 

Don’t  Include 

—  Select  - 

 -^Siject  n 

-  Select  - 

-  Select  - 

-•  Select  - 

LA- : 

 Sele.rN-  

Total  Lifecycle  Cost 

Include 

_ 5 _ - 

%i  299 

^’TOCT" 

""53  "950"" 

■  ■  ■  ■  ■  IIUJ  *■■■■■ 

. 

. Alternative  Selection  and  Simple  Evaluation 


Cost  information  for  cost-benefit  analyses. 
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Weapon-mountable  Green  Laser  Application 

Alternative  Comparison  DST  Outputs 


Criteria 

Criteria 

Weights 

Helios 

Dazzlei 

PD  G-105 
Dazzlei 

Mini  Green 

GBD  3 

BTG2S 

Co  info  it 
Class  3 

GL  V5 

S532A 

Weight 

0.167 

25 

100 

75 

50 

50 

75 

50 

75 

Output  Powei 

0.083 

100 

50 

50 

75 

25 

25 

25 

25 

Beam  Divergence 

0.125 

75 

50 

100 

100 

50 

50 

50 

50 

Rail  Mountahility 

0.208 

100 

100 

100 

50 

25 

25 

25 

25 

Range 

0.167 

25 

25 

50 

100 

75 

25 

75 

75 

Batteiy 

0.083 

75 

100 

100 

100 

100 

100 

100 

100 

Vendor  Reliability 

0.167 

100 

50 

50 

50 

50 

50 

50 

Total  Value  Score 

1.000 

69.8 

68.8 

r(  75.0) 

70.8 

51.0 

46.9 

51.0 

55.2 

Decision  Matrix 


Highest  Scoring 


Total  Value  Score  Comparison  Chart 


Total  Value  Score  Comparison  Stacked  Column  Chart 
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Weapon-mountable  Green  Laser  Application 

Sensitivity  Analysis  DST  Outputs 


Slider  bars 
allow  the  user  to 
adjust  the 
weights  of 
multiple  criteria 
to  see  the 
effects  on  the 
alternative  total 
value  scores. 


Return  Weights  to  Original 
Values 


Ciiteria 

Original 

Weight 

Rescaled 

Weight 

Weight 

0.167 

0.130 

x 

Output  Power 

0.083 

0.140 

Beam  Divergence 

0.125 

0.080  1 

jDJ  ► 

Rail  Mountability 

0.208 

0.220 

Range 

0.167 

0.140  ' 

< 

Battery 

0.083 

0.100 

Vendor  Reliability 

0.167 

0.190 

v  y 

Dynamic 
comparison 
chart  shows 
the  effects  on 
alternative 
total  value 
scores  from 
criterion 
weight 
changes. 


Alterative  Comparison  Chart  with  Dynamic  Sliders  Bars  For  Sensitivity  Analysis  of  the  Criteria  Weights 


Laser  Range  Sensitivity  Analysis  Chart 
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Weapon-mountable  Green  Laser  Application 

Cost-Benefit  DST  Output 


Cost-benefit  chart  shows  each 
alternative’s  total  value  score  versus  its 
cost. 


Total  Value  Score  versus  Total  Lifecycle  Cost 


♦  Helios  Dazzler 

■  PD/G-105 
Dazzler 

Mini  Green 
=<  GBD  3 
X  BTG  2S 

♦  Comfort  Class  3 
+  GLV5 

-  S532A 


Cost-Benefit  Chart 


Shows  the  decision 
maker  the  added  benefit 
provided  by  increased 
cost  and  readily 
identifies  dominated 
alternatives  (alternatives 
that  provide  less  value 
but  cost  more  than 
another  alternative). 
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Human  Systems  Integration  (HSI)  Analysis 

Overview 


Purpose:  to  integrate  manpower,  personnel,  training,  human  factors  engineering, 
habitability,  survivability,  and  environment,  safety  and  occupational  health 
considerations  into  the  system  acquisition  process. 


HSI  provides  a  human  performance 
perspective  of  system  integration. 

HSI  analyses  explicitly  evaluate  and 
integrate  the  seven  functional 
human-related  areas  (HSI  domains) 
into  the  systems  acquisition  process. 

Developing  a  tool  capable  of: 

-  Supporting  comparisons  of 
alternatives  from  an  HSI  perspective. 

-  Developing  customized 
questionnaires  to  provide  item- 
specific  feedback  and  analysis 
of  item  effectiveness  from  an  HSI 
perspective. 


Screenshot  from  Prototype  Tool 


•  MAGTF  C2  Test  Branch,  US  Marine  Corps,  has  already  seen  and 
requested  a  copy  of  the  tool. 


Consideration  of  HSI  factors  is  critical  due  to  short  acquisition  timelines  and 
the  fact  that  REF  systems  will  be  directly  integrated  into  operational  units. 
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Information  Technology  Management  Analysis 

Overview 


Purpose:  to  develop  a  framework  to  assess  and  redesign  core  business  processes 
with  information  systems  to  support  potential  process  improvement  within  REF. 


•  Knowledge  Value  Added  (KVA)  analysis. 

-  Views  organization  as  portfolio  of  knowledge  assets  deployed  to  create 
value  (people  and  systems). 

-  Assesses  value  of  intellectual  capital 
and  information  technology. 

-  Provides  a  value  benchmark  for 
comparisons  across  all  processes. 

-  Permits  tracking  of  value  over  time. 

•  Deliverables. 

-  A  baseline  model  to  estimate  the 
relative  returns  on  investment  (ROIs) 
for  REF  processes. 

-  Alternative  process  models,  job  descriptions, 
and  a  table  of  the  improvements  in  relative  returns  for  each  model. 


GaussSoft  Software  Used  for  Analyses 


Analyses  will  provide  valuable  input  for  identifying  potential  processes  for 
improvement  and  for  examining  the  value  of  potential  strategic  options. 


14  June  2007 


REF  Analytical  Support  MORSS  Briefing 


26 


Rapid  Ordering  System  (ROS)  Integration 

Overview 


Purpose:  to  streamline  the  contracting  system  used  by  the  REF  through  the 

integration  of  the  Rapid  Ordering  System  (ROS). 

•  ROS  vision. 

-  On-line  graphical  user  interface  (GUI)  for  contracting. 

-  Accesses  every  potential  vendor  interested  in  supporting  DoD. 

-  Accesses  electronic  vendor  catalogs  directly  for  data  searching. 

-  Prepares  all  contract  ordering  documents  (narratives  and 
forms). 

-  Connects  IDIQ  contracts  from  all  Government  contracting 
offices. 

-  Is  accepted  and  incorporated  into  the  entire  DoD  contracting 
process. 

-  Creates  automated  re-ordering. 

-  Automated  closeout. 


Implementation  of  the  ROS  within  the  REF  has  the  potential  to  shorten  the 
acquisition  timeline  significantly  and  reduce  workload. 
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Rapid  Ordering  System  (ROS)  Integration 

Sample  ROS  Interfaces 


Vendor  Items 


Approved  Items  Add  Approved  Vendor  Items 


Save  Delete 


5000- 

0001 


5000- 

0002 


5000- 

0003 


5000- 

0005 


5000- 

0006 


5000- 

0007 


5000- 

0009 


5000- 

0013 


5000- 

0014 


IDAS  Integrated 
Demounted  Armor 
System,  Small  - 
Marpat 

IDAS  Integrated 
Dismounted  Armor 
System,  Medium  - 
Marpat 

IDAS  Integrated 
Demounted  Armor 
System,  Large  - 
Marpat 

IDAS  Integrated 
Dismounted  Armor 
System,  Small  -  3 
Color  Desert 
IDAS  Integrated 
Dismounted  Armor 
System,  Medium  - 
3  Cobr  Desert 
IDAS  Integrated 
Demounted  Armor 
System,  Large  -  3 
Cobr  Desert 
S-SAPI  Secondary 
Smal  Arms 
Protective  Insert  - 
Coyote  Brown 
IDAS(G)  Integrated 
Demounted  Armor 
System  (Gunner), 
Small  -  Army  Digital 
Green  (ACU) 
IDAS(G)  Integrated 
Demounted  Armor 
System  (Gunner), 


Vendor: 

My  Vendor: 

Item  Number: 
Item  Description: 

Vendor 's  Item 
Number: 

Vendor 's 
Description: 

Item  Size: 

Unit  Price: 


Med-Eng 


13 


Unit  of  Measure:  EACH 


My  Vendor 


Extended 

Description: 


5000-0009 


S-SAPI  Secondary  Small  Arms  Protective  Insert 


T o  combat  the  increased  injuries  from 
lateral  gunshot  wounds,  the  S-SAPI 
system  uses  in-service  X-SmaB  OTV  SAPI 
and  commercial  6"  x  8"  plates  in  a 
custom  pouch.  The  pouch,  made  of 
500d  cordura  uses  standard  MOLLE 
attachment  bops  fits  aN  sizes  of  OTV  as 
wel  as  other  SF  vests.  The  S-SAPI 


(none)  vj 


$189.00 


Co  yeet  Brawn 

<ACU> 


Manufacturer 

Name: 

Photo  URL: 
View  Photo: 


http://www.med-eng.com/fp/ros/photos/S-SAF 

http://www.med-enq.com/fp/ros/photos/S- 

SAPI.idq 

Specification  Url:  http://vAwy.med-eng.com/fp/ros/pdfs/SAPI_Apn 

View  http :// '.wav  .med- 


-  1  -  RFP  Entry 

Requests 


□© 

Add 


Project 

RFP  Name 

|  Created 

RFP  Id 

Marine  Undergarment  Purchase 

Quantico  Undergarment  Purchase  2 

5/12/2006 

39 

Aug3Vend2 

Frank  Test 

10/4/2006 

92 

Ron's  Project 

Ron's  Test 

93 

l 

Request  Details 

Save  Delete  Notify 

Procurement 

Type: 

Project: 

Created  By: 

RFP  Name: 


Extended 

Description: 


Status: 

Created  Date: 

Request  Date: 

Deliver  By  Date: 

Respond  By 
Date: 


a 


Select  value... 


Quantico  Undergarment  I 


5/12/2006 


J5/12/2006 


5/12/2006 


8/1/2006 


Calendar 

Calendar 

Calendar 


Issuer 

Contact: 

Issuer 

Contact 

Email: 

Issuer 

Contact 

Phone: 

Ship  To: 

Ship  To 
DODAAC: 
Ship  To 
Contact: 


Internal 

Notes: 


Is  Price 
Factor  ?: 


John  Hernandez 


nul@nul.gov 


(703)  432-3305 


PM  ICE,  MARCORSYSCOM 
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General  Analysis  Methodology 

Overview 
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Conclusions 


•  This  collaborative,  multi-disciplinary  effort  has  resulted  in  a  series  of 
techniques  and  tools  that  will  be  valuable  to  the  REF  and  that  may  be 
leveraged  for  other  organizations. 

•  Will  benefit  the  REF  by  providing: 

-  An  iterative,  quick  turn-around,  standardized  systems  analysis 
methodology  with  supporting  tools  and  analyses. 

-  An  analysis  guidebook  to  provide  all  REF  analysts,  regardless  of  their 
analytical  background,  a  common  framework  for  REF  analyses. 

-  A  decision  support  tool  to  facilitate  rapid,  quantitative  alternative 
comparisons  while  considering  critical  evaluation  measures. 

-  A  means  to  consider  critical  HSI  factors  during  requirements  analysis 
and  project  evaluation  after  fielding. 

-  Valuable  input  for  identifying  potential  processes  for  improvement  and 
for  examining  the  value  of  potential  strategic  options. 

-  A  rapid  contracting  system  to  shorten  the  acquisition  timeline  and 
reduce  workload. 

•  Many  portions  of  this  work  have  already  been  well-received. 

-  Human  systems  integration  (HSI)  tool. 

-  Tailored  methodology. 

-  Concept  of  the  Rapid  Ordering  System. 
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2.0  Requirements  Analysis  /  Project  Validation 

Inputs  to  Outputs 


INPUTS 

PROCESS 

OUTPUTS 

A.1  Need 
Document 


(10-Liner,  ONS, 
JUONS) 


C.2 

TM  Potential 
Solutions 


H 


Technical  COA 
Analysis  Results 


2.0 

Requirements 
Analysis  /  Project 
Validation 


J 

TM  White  Paper 


K 


TM  Board  Slide  / 
Briefing 


A.2 

QA’d  Need 
Documerrt 

B.1 

Requirements 
Quad  Chart 

C.1 

Partial  Solution 
Set___ 

D 

REF  Priority 

'  "  "  . 

E 

Supporting 

Requirements 

Information 

F 

Requests  for 
Information 
(RFIs) _ 

G 

Correspondence 

History 

Database  Entries 
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2.0  Requirements  Analysis  /  Project  Validation 

Sub-processes 


2.1  QA  Need  Document 
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2.0  Requirements  Analysis  /  Project  Validation 

Discussion  (Slide  1  of  4) 


•  Purpose:  To  transform  an  initial  need  identification  into  a  thorough  set  of  requirements  by  refining 
the  initial  need  documentation  and  conducting  a  detailed  mission  (need)  analysis. 

•  Rationale:  As  with  the  initial  identification  of  the  need,  this  essential  step  in  defining  the  problem 
and  determining  the  requirements  is  critical  to  the  success  of  the  project.  The  requirements 
identified  in  this  process  is  the  foundation  upon  which  the  ultimate  recommendation  of  a  solution 
and  the  implementation  plan  will  be  based. 


•  Inputs:  The  primary  input  to  this  process  is  a  10-Liner,  Operational  Need  Statement  (ONS),  or  a 
Joint  Use  Operational  Need  Statement  (JUONS).  Inputs  could  also  include  documentation  from  the 
REF  Technology  Management  team  as  they  vet  ‘good  ideas’,  new  technologies,  insert 
technologies,  or  command-directed  technologies. 


•  Sub-process  Discussion: 


-2.1  QA  Need  Document:  In  this  step,  the  team  receives  a  need  identification  from  the  field  (10- 
Liner,  ONS,  JUONS)  and  reviews  it  for  completeness.  If  the  documentation  is  missing  critical 
information,  the  team  returns  it  to  the  originator  for  clarification.  Some  information  is  not 
required  to  be  complete  in  the  Need  Document  (e.g^  organizational  concept,  procurement 
objective,  support  requirements,  and  availability). 'The  output  is  a  QA’d  need  document. 


-2.2  Conduct  Mission  Analysis:  This  is  a  very  complex  step  and  includes  numerous  sub-steps  to 
ensure  that  the  initial  need  statement  is  accurately  transformed  into  a  thorough  set  of 
requirements.  A  discussion  of  these  sub-steps  is  essential  to  understand  this  process. 


-  2.2.1  Stakeholder  and  Background  Analysis:  The  team  begins  this  step  immediately  and 
continues  to  execute  it  throughout  the  entire  process.  A  stakeholder  can  be  defined  as 
anyone  with  a  vested  interest  in,  or  influence  upon,  the  problem.  The  team  will  identify  and 
interview  stakeholders  for  almost  every  aspect  of  Mission  Analysis.  Because  time  is  a  limited 
commodity  for  all  stakeholders,  the  team  must  put  thought  into  the  information  they  need 
before  they  begin  their  interviews.  Key  stakeholders  win  include  the  unit,  threat  subject 
matter  experts  (SMEs),  other  operational  SMEs,  REF  personnel,  project  managers  (FTVIs), 
maintenance  and  sustainment  SMEs,  industry,  and  other  personnel  who  can  help  provide  or 
clarify  required  information.  Additionally,  the  team  will  check  with  DAMO-CIC  to  determine  if 
they  are  working  on  a  similar  ONS  prior  to  an  AR2B.  In  conjunction  with  interviewing 
stakeholders,  the  team  will  also  conduct  background  research  to  clarify  their  information. 

The  key  outputs  of  this  step  are  a  correspondence  history  and  the  information  provided  by 
the  stakeholders  and  through  the  background  research.  In  some  cases,  if  a  PM  solution  has 
been  identified  to  meet  the  need,  this  process  can  end  with  transition  to  the  PM. 
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2.0  Requirements  Analysis  /  Project  Validation 

Discussion  (Slide  2  of  4) 


•  Sub-process  Discussion  (continued): 

-2.2  Conduct  Mission  Analysis  (continued): 

-  2.2.2  Capability  Gap  Analysis:  The  team  will  refine  the  capability  gaps  identified  by  the  REF 
FWD  team  (if  provided)  and  identify  applicable  sub-capability  gaps  as  well.  The  team  should 
try  to  identify  the  frequency  with  wnich  the  gap  occurs  and  tne  impact  of  the  gap  on  the  unit. 

-  2.2.3  Operational  Environment  Analysis:  The  team  will  describe  the  operational  environment 
in  which  the  solution  will  be  used,  identify  ROE  and  other  legal  considerations,  and 
understand  the  threat  and  how  the  threat  might  defeat  and/or  adapt  to  the  employment  of  the 
solution. 

-  2.2.4  Operational  Opportunity  and  Context  Analysis:  This  analysis  captures  how  the  potential 
solution  may  be  employed,  in  terms  of  tactics,  techniques  ana  procedures  (TTPs),  identifies 
potential  drawbacks/penalties  of  the  system,  identifies  other  systems  with  which  the  potential 
solution  will  have  to  interoperate,  identifies  how  often  the  potential  system  will  be  available  to 
the  unit  for  use,  and  how  often  situations  will  occur  for  which  the  system  will  provide  benefit. 
The  latter  two  analyses  help  the  REF  determine  if  the  fielded  solution  will  actually  provide 
significant  benefit  to  the  unit,  versus  only  occasional  benefit.  These  results  should  be 
balanced  with  the  frequency  and  impacts  of  the  gap  itself  to  help  the  REF  prioritize  the 
current  requirement  with  other  competing  requirements. 

-  2.2.5  Fielding  and  Sustainment  Analysis:  This  analysis  captures  the  number  of  systems  that 
are  required  and  whom  they  will  be  fielded  to  (basis  of  issue  -  number  by  echelon),  the 
expect  logistical  footprint  required  to  support  the  system  (manpower,  spares,  facilities, 
special  equipment)  and  an  initial  estimate  of  the  lifecycle  cost  (procurement,  maintenance, 
sustainment,  etc.)  of  a  potential  solution  (to  help  identify  the  funding  source)  and  an  estimate 
of  the  fielding  schedule. 

-  2.2.6  Operational  Feasibility  Analysis:  This  analysis  captures  the  organization,  training,  leader 
education,  and  personnel  impacts  of  fielding  the  system  and  includes  the  human  factors 
issues  (comfort,  usability,  etc.)  associated  with  the  fielded  systems’  impacts  on  the  users, 
leaders,  and  maintainers. 

-  2.2.7  Functional  Analysis:  This  analysis  defines  in  greater  detail  the  required  functions  of  the 
system  in  terms  of  what  the  system  must  do  in  order  to  mitigate  the  capability  gap  and  satisfy 
the  unit’s  need.  This  is  primary  step  for  defining  the  requirements  of  tne  potential  materiel 
solution  and  provides  the  basis  for  alternative  identification  and  the  evaluation  framework 
that  will  be  used  to  compare  the  alternatives.  It  is  important,  when  defining  the  functions,  to 
define  them  independently  of  potential  solutions  already  being  considered? 
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2.0  Requirements  Analysis  /  Project  Validation 

Discussion  (Slide  3  of  4) 


•  Sub-process  Discussion  (continued): 

-2.2  Conduct  Mission  Analysis  (continued): 

-  2.2.8  Evaluation  Framework  Design:  This  step  should  flow  directly  from  the  functional 
analysis  and  the  stakeholder  analysis.  The  objectives  of  the  stakeholders  should  be 
combined  with  the  required  functions  to  define  the  desired  performance  by  function.  The 
team  will  then  derive  criteria  from  those  objectives  that  will  be  used  to  measure  and  evaluate 
alternative  performance.  Criteria  come  from  the  system  attributes  defined  in  the  10-Liner, 
measures  suggested  by  stakeholders,  and  other  measures  developed  by  the  team.  Criteria 
must  be  developed  in  such  a  way  that  is  not  specific  to  a  particular  solution  so  that  they  can 
be  used  to  evaluate  any  potential  alternative.  Criteria  can  either  be  natural  (a  clearly  defined 
and  understood  scale}1ike  pounds)  or  constructed  (created  specifically  to  measure  the 
performance  of  a  particular  function/objective,  such  as  a  5-star  rating  for  safety).  Once  a  list 
of  criteria  has  been  established,  the  team  must  weight  the  criteria  according  to  the 
importance  of  the  primary  stakeholders,  so  that  comparisons  can  be  made.  The  evaluation 
framework  will  be  refined  and  completed  later  by  the  Project  Officer. 

-  2.2.9  Initial  Alternative  Identification:  In  this  step,  the  team  will  compile  a  set  of  potential 
solutions,  which  will  come  from  the  unit,  other  stakeholders,  and  other  sources.  The  team 
should  consider  non-materiel  alternatives  within  the  DOT_LPF*  domains  as  well. 

-2.3  Assign  REF  Priority:  The  requirements  team  should  assign  a  priority  (routine,  essential,  or 
urgent)  fo  the  requirement  based  primarily  upon  unit/REF  FWD  recommendation,  but  also  upon 
the  Mission  Analysis  conducted  as  part  of  this  process. 


*DOT_LPF:  Doctrine,  Organization,  Training,  Leader  Education,  Personnel,  Facilities  (Materiel  omitted) 


•  Potential  Tools  and  Techniques: 

-Stakeholder  Analysis:  The  key  to  the  success  of  this  process  is  the  interaction  between  the  REF 
analysis  teams  and  key  stakeholders. 

-TRADOC  ARCIC  Current  Force  Capability  Gap  Documentation:  Provides  a  description  of  the 
currently-approved  capability  gaps,  sub-capability  gaps,  and  potential  solutions  for  mitigating 
the  gaps. 

-Human  Systems  Integration  (HSI)  Tool:  Tool  developed  by  TRAC-MTRY  to  assist  in  the 
identification  of  potential  HSl  ana  DOTMLPF  issues  associated  with  fielding  a  materiel  solution. 

-System  Decomposition:  A  technique  to  decompose  the  required  solution  into  its  component 
parts,  structure,  and  functions.  This  is  a  good  technique  for  gaining  an  understanding  of  the 
system  and  its  key  elements. 
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Discussion  (Slide  4  of  4) 


•  Potential  Tools  and  Techniques  (continued): 

-System  Context  Diagram:  A  technique  used  to  place  the  system  and  its  sub-systems  in  the 
context  of  the  operational  environment,  which  is  helpful  for  identifying  interoperability  issues, 
potential  TTPs,  and  echelon  of  system  employment. 

-Legal  Advice:  Army  lawyers  can  help  identify  unique  legal  requirements  for  certain  types  of 
systems  under  consideration. 

-Functional  Decomposition:  Techniques,  such  as  functional  hierarchies  and  functional  flow 
diagrams,  used  to  decompose  the  need  into  required  functions. 

-Objectives:  A  method  of  thinking  that  translates  required  functions  into  desires  concerning 
performance  that  can  help  lead  directly  into  the  criteria  or  measures  for  alternative  evaluation. 

-Value  Measures:  Techniques  for  developing  criteria  to  capture  the  value  of  the  key  aspects  of 
alternative  performance  for  evaluation  and  comparison  purposes. 

-Value  Hierarchy:  Technique  used  to  capture  the  evaluation  framework  that  starts  with  the 
functions  and  flows  into  objectives  and  then  criteria,  or  value  measures. 


•  Outputs: 

-QA’d  Need  Document. 

-Requirements  Quad  Chart:  Pre-defined  REF  document  that  includes  a  succinct  description  of 
the  problem  (this  should  be  a  more  complete,  revised  problem  statement  that  builds  upon  the 
initial  problem  statement  included  in  the  Need  Document),  a  photo  of  the  potential  solution  (if 
identified),  a  description  of  the  system  (either  explicitly,  if  a  solution  has  been  identified,  or  in 
terms  of  requirements  if  a  solution  has  not  been  identified),  the  applicable  TRADOC  ARCIC 
Current  Force  capability  gap(s)  and  sub-capability  gap(s),  proposed  and  recommended  courses 
of  action  for  the  procurement  of  the  solution,  the  requesting  activity,  identification  of  similar 
items  REF  may  have  considered  in  the  past,  an  initial  sustainment  strategy,  and  a  potential 
vendor  (if  identified). 

-Partial  Solution  Set:  A  set  of  potential  materiel  solutions  described  in  an  Excel  spreadsheet. 

-REF  Priority. 

-All  of  the  supporting  Requirements  Analysis  /  Project  Validation  products. 

-Unanswered  Request  for  Information  (RFIs). 

-Stakeholder  Correspondence  History:  A  log  of  the  critical  discussions  between  the 
requirements  team  and  key  stakeholders. 

-Required  Database  Entries. 


14  June  2007 


REF  Analytical  Support  MORSS  Briefing 


37 


5.0  Solution  Set  Refinement  and  Analysis 
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5.0  Solution  Set  Refinement  and  Analysis 
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5.0  Solution  Set  Refinement  and  Analysis 

Discussion  (Slide  1  of  5) 


•  Purpose:  To  identify,  analyze,  and  compare  potential  solutions  to  the  stated  requirement  and  to 
recommend  the  best  potential  solution. 

•  Rationale:  This  step  builds  upon  the  foundation  provided  by  the  Requirements  Analysis  to  identify 
tne  pest  solution.  Thus,  this  process  will  determine  the  system  that  gets  fielded  to  the  operational 
unit,  and  is  therefore  central  to  the  REF  process.  It  is  critical  that  this  step  be  performed  well  to 
provide  the  best  performing  solution  at  the  best  cost  that  will  meet  the  unit’s  needs. 


•  Inputs: 

-Project  Validation  Documents: 

-  QA’d  Need  Document. 

-  Signed  Requirements  Quad  Chart. 

-  Partial  Solution  Set. 

-  Director’s  Guidance. 

-  Supporting  Requirements  Information. 

-  Unanswered  Requests  for  Information. 

-  Correspondence  History. 

-Technology  Management  Documents: 

-  TM  Potential  Solutions. 

-  Technical  COA  Analysis  Results. 

-  TM  White  Paper. 

-  TM  Board  Slide/Briefing. 


•  Sub-process  Discussion: 

-5.1  Assign  Project  Officer  Team:  In  this  step,  the  requirement  is  received  by  the  project  team 
and  a  Project  Officer  is  identified.  Other  team  members  are  given  a  heads  up  concerning  the 
new  requirement  so  that  parallel  planning  can  be  executed.  This  step  also  involves  scoping  the 
effort. 

-5.2  Refine  Requirements  Products:  In  this  step,  the  Project  Officer  examines  the  documentation 
from  the  Requirements  Analysis  and  refines  any  information  the  Project  Officer  deems 
insufficient  or  missing.  This  step  also  includes  pursuing  any  unanswered  RFIs. 

-5.3  Identify  Potential  Alternatives:  This  involves  a  thorough  process  of  identifying  all  possible 
solutions  to  meet  the  unit  need.  The  Project  Officer  can  accomplish  this  step  by  contacting  old 
POCs  for  like  capabilities,  coordinating  with  acquisition  PMs  for  capabilities  current  part  of... 
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5.0  Solution  Set  Refinement  and  Analysis 

Discussion  (Slide  2  of  5) 


•  Sub-process  Discussion  (continued): 

-5.3  Identify  Potential  Alternatives  (continued):  ...  of  their  programs,  researching  previous  REF 
activities  for  like  solutions,  searching  existing  technologies  through  a  variety  of  search  engines, 
and  considering  non-materiel  (DOT  LPF)  alternatives.  The  project  officer  will  also  have  input 
from  the  unit,  the  TM  COA  ana  lysis  "(if  conducted),  and  the  Reguirements  Analysis  team.  This 
step  may  result  in  passing  the  project  to  an  appropriate  PM,  if  one  has  been  identified. 

-5.4  Analyze  Alternatives:  This  is  a  very  complex  step  and  includes  numerous  sub-steps  to 
methodically  analyze  the  set  of  potential  alternatives.  A  discussion  of  these  sub-steps  is 
essential  to  understand  this  process. 

-  5.4.1  Refine  Evaluation  Framework:  A  detailed  discussion  of  this  task  can  be  found  in  the 
discussion  for  process  2.2.8.  Here,  the  Project  Officer  refines  that  original  analysis  by 
ensuring  that  a  complete  framework  is  defined.  This  step  may  have  only  been  partially 
completed  during  the  reguirements  analysis  and  should  be  completed  here.  Additionally,  the 
identification  of  new  alternatives  will  often  reguire  changes  to  the  framework. 

-  5.4.2  Screen  Out  Infeasible  Alternatives:  As  part  of  the  evaluation  framework  design,  and  at 
other  parts  of  the  Reguirements  Analysis,  the  analysts  may  have  identified  constraints  upon 
the  system.  These  are  conditions  that  the  system  must  meet  in  order  to  be  feasible  solutions. 
These  may  come  from  the  unit  or  other  stakeholders,  such  as  the  REF  Director.  If  these  have 
not  been  defined  previously,  they  should  be  compiled  here.  Once  a  complete  list  of  system 
constraints  has  compiled,  the  analyst  should  remove  from  consideration  any  alternative  that 
does  not  meet  the  constraints.  Prior  to  this  step,  alternatives  should  have  been  developed 
without  considering  these  constraints. 

-  5.4.3  Conduct  Market  Research:  In  this  step,  the  Project  Officer  conducts  research  on  each  of 
the  remaining  feasible  alternatives  to  determine  as  much  information  about  each  of  the 
alternatives  as  possible.  At  a  minimum,  the  Project  Officer  must  be  able  to  characterize  the 
alternative  performance  for  each  of  the  criteria  within  the  evaluation  framework.  If 
information  concerning  those  alternatives  is  incomplete,  the  Project  Officer  must  either 
estimate  the  performance  or  change  the  criterion  in  a  way  that  can  be  evaluated  for  all 
alternatives.  This  may  reguire  changing  the  criterion  from  a  natural  measure,  that  may  not  be 
possible  to  estimate,  to  a  subjective  measure,  that  is  more  flexible.  Included  in  this  step  is  a 
refinement  of  the  lifecycle  cost  estimate  for  each  alternative. 

-  5.4.4  Conduct  Modeling:  This  step  is  an  optional,  but  often  used,  process  used  to  estimate 
the  performance  of  an  alternative  for  one  or  more  criteria.  Models  can  range  from  simple 
mathematical  approximations  to  large  simulation  models.  The  specific  type  of  model  used 
depends  greatly  on  the  solution  set,  the  criteria  to  be  estimated,  and  the  time  available  to 
conduct  the  analysis. 
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5.0  Solution  Set  Refinement  and  Analysis 

Discussion  (Slide  3  of  5) 


•  Sub-process  Discussion  (continued): 

-5.4  Analyze  Alternatives  (continued): 

-  5.4.5  Conduct  Testing  or  Experimentation:  This  is  an  optional  step  used  to  estimate 
alternative  performance  for  one  or  more  criteria.  This  process  usually  involves  the 
procurement  of  one  or  more  of  the  candidate  alternatives  and  consists  of  placing  the  system 
under  the  appropriate  operating  conditions  and  evaluating  the  performance.  The  use  of  this 
step  depends  upon  the  time  availably  system  availability  and  cost,  and  the  testing  / 
experimentation  capabilities  of  the  REF,  and  can  occur  after  fielding  candidate  systems  to  the 
unit.  This  process  can  potentially  be  performed  by  an  testing  /  experimentation  agency,  such 
as  the  US  Army  Test  and  Evaluation  Command  (AtEC). 

-  5.4.6  Assess  Alternative  Performance:  This  step  consists  of  compiling  the  alternative 
performance  for  each  alternative  with  respect  to  each  criterion  into  a  single  product  for 
comparison.  If  quantitative  analysis  is  not  going  to  be  used,  this  set  of  data  will  be  examined 
in  the  next  process  to  determine  the  best  alternative.  If  quantitative  analysis  is  to  be  used, 
the  raw  data  must  be  converted  a  common  scale  (often  called  value  scoring)  for  comparative 
purposes,  allowing  an  ‘apples-to-apples’  comparison. 

-5.5  Compare  Alternatives: 

-  5.5.1  Identify  Relative  Alternative  Performance:  The  endstate  of  this  process  is  a  rank- 
ordered  list  of  the  potential  solutions  based  upon  their  expected  performance.  If  quantitative 
analysis  is  not  to  be  used,  the  Project  Officer  must  look  at  the  raw  performance  data  and 
manually  rank  order  the  alternatives  based  on  that  data  and  based  upon  comparisons  with 
the  requirements  defined  earlier  in  the  process.  If  quantitative  analysis  is  being  used,  the 
Project  Officer  will  use  the  criteria  weights  in  combination  with  the  common  scales  (value 
scales)  for  the  alternative  performance  to  calculate  a  total  value  score  for  each  alternative. 

The  alternatives  will  then  be  ranked  according  to  those  value  scores. 

-  5.5.2  Conduct  Sensitivity  Analyses:  Since  much  of  the  analysis  done  to  rank  the  alternatives 
is  subjective  in  nature  (regardless  of  whether  the  analysis  is  quantitatively-  or  qualitatively- 
based),  some  examination  of  the  assumptions  that  lecfto  the  recommended  alternative  must 
be  conducted.  For  quantitative  analyses,  sensitivity  analyses  often  involve  examining  the 
criteria  weighting  in  order  to  identify  those  cases  wnen  small,  realistic  changes  to  the  weights 
change  the  recommended  alternative.  Additionally,  sensitivity  analyses  should  also  examine 
how  plausible  changes  to  the  estimated  performance  can  affect  the  final  recommended 
alternative. 
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Discussion  (Slide  4  of  5) 


•  Sub-process  Discussion  (continued): 

-5.5  Compare  Alternatives  (continued): 

-  5.5.3  Conduct  Trade-off  Analyses:  This  process  involves  examining  the  trade-offs  between 
alternative  performance  and  key  selected  parameters,  such  as  cost,  availability,  etc.,  which 
are  treated  as  independent  variables.  It  usually  consists  of  plotting  the  alternative 
performances  (or  total  value  scores)  against  the  independent  variable  and  determining  how 
much  performance  (or  value)  is  gaineoby  added  cost,  longer  delivery  times,  etc.  It  is 
important,  for  plots  of  performance  versus  the  selected  key  parameter,  that  the  parameter  is 
not  also  included  in  the  calculation  of  the  overall  alternative  performance,  which  would  lead 
to  double-counting. 

-5.6  Recommend  Alternative:  Based  upon  the  overall  alternative  comparison  results,  sensitivity 
analyses,  and  trade-off  analyses,  the  Project  Officer  makes  a  recommendation  for  a  particular 
solution.  In  some  cases  in  quantitative  analyses,  the  recommended  alternative  may  be  different 
from  the  highest  scoring  alternative  based  upon  the  results  of  the  sensitivity  or  trade-off 
analyses,  or  based  upon  other,  non-guantifiable  aspects.  Quantitative  results  should  never  be 
allowed  to  override  common  sense;  however,  the  Project  Officer  should  be  able  to  defend 
thoroughly  any  recommendation  and  should  be  prepared  to  show  key  analysis  results  to  the 
decision-maker. 


•  Potential  Tools  and  Technigues: 

-Value-focused  Thinking:  The  same  tools  discussed  in  Process  2.0,  such  as  objectives,  value 
measures,  value  scores  and  curves,  and  the  value  hierarchy. 

-Technology  Search  Tools:  Tools  such  as  Google  and  more  specific  search  engines,  designed  to 
identify  potential  alternatives. 

-Constraints  and  Feasibility  Matrices:  Technigues  and  tools  to  define  system  criteria  and  to 
facilitate  alternative  screening. 

-Systems  Comparison  Matrix:  A  REF-designed  product  in  an  Excel  spreadsheet  used  to  capture 
the  key  performance  information  of  each  of  the  alternatives. 

-Models  and  Simulation:  A  wide  range  of  tools  and  technigues  to  estimate  alternative 
performance  under  operational  conditions. 

-Experimental  Design:  Techniques  to  systematically  vary  key  alternative  factors  during 
experimentation  to  ensure  that  the  team  can  make  statistically  valid  estimates  of  alternative 
performance  based  upon  the  experimental  results. 
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5.0  Solution  Set  Refinement  and  Analysis 

Discussion  (Slide  5  of  5) 


•  Potential  Tools  and  Techniques  (continued): 

-Sensitivity  Analyses:  Techniques  to  examine  the  sensitivity  of  criteria  weights  and  alternative 
scores. 

-Cost-benefit  Plots:  Plots  of  alternative  performance  (or  total  value  scores)  against  cost, 
availability,  etc.,  used  for  trade-off  analyses. 

-Decision  Support  Tool:  Developed  by  TRAC-MTRY  to  facilitate  simple  and  more  complex  criteria 
weighting  techniques,  simple  and  more  complex  alternative  scoring  techniques,  and  to  provide 
alternative  comparison,  sensitivity  analysis,  and  cost-benefit  charts. 


•  Outputs: 

-Project  Quad  Chart:  Pre-defined  REF  document  of  the  same  format  as  the  Requirements  Quad 
Chart.  This  chart  should  have  all  information  concerning  the  system  characteristics,  photo, 
cost,  schedule,  etc.,  that  may  have  been  incomplete  on  the  Requirements  Quad  Chart. 

-Solution  Set  Information  Worksheet:  A  spreadsheet  containing  detailed  information  concerning 
every  potential  solution  considered  during  the  process. 

-Alternatives  Analysis  Results:  All  of  the  analysis  products,  such  as  charts,  spreadsheets, 
models,  etc.,  resulting  from  the  Solution  Set  Refinement  and  Analysis  process. 
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